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Background: Canada has two health technology assessment (HTA) agencies responsible 
for oncology drug funding recommendations: the Institut National d'Excellence en Sante et 
Services Sociaux (INESSS) for the province of Quebec and the pan-Canadian Oncology Drug 
Review for the rest of Canada. The objective of the research was to review and compare the 
recommendations of these two agencies alongside an international comparator - the National 
Institute for Health and Care Excellence (NICE) in the United Kingdom - with respect to their 
recommendations records and the influence of clinical and cost-effectiveness evidence on the 
recommendations. 

Methods: Recommendations were identified from the three agencies from January 1, 2002 to 
June 1, 2013. Recommendations were limited to five cancer sites (lung, breast, colon, kidney, 
blood) and to metastatic/advanced settings. Descriptive analyses examined the frequency 
of positive recommendations and factors related to a positive recommendation. For each 
recommendation, only publicly available information posted on the agency website was used 
to abstract data. 

Results: There was a wide variation in the rate of positive recommendations, ranging from 
48% for NICE to 95% for Canada's national process (among the 74% of its recommendations 
that were publicly posted). Interagency agreement was low, with full agreement for only six 
of the 14 drugs commonly reviewed by all three agencies. Evidence of a survival gain was not 
necessary for a positive recommendation; progression-free survival was acceptable. Different 
approaches were taken when addressing unacceptable cost-effectiveness. NICE was most likely 
to yield a negative recommendation on these grounds, whereas Canada's national process was 
most likely to yield a positive recommendation with a required pricing arrangement. 
Conclusion: In this analysis, the primary reason for the observed divergence between agency 
recommendations appeared to be the availability of mechanisms in each jurisdiction to address 
cost-effectiveness subsequent to the HTA assessment process. Furthermore, caution is needed 
when interpreting cross-agency comparisons between HTA agencies, as recommendations may 
not correspond directly to subsequent funding decisions and actual patient access. This may 
be a concern, given the high international profile of assessments conducted by the reviewed 
HTA agencies. 

Keywords: reimbursement, decision-making, oncology, health technology assessment, funding 
decisions, metastatic/advanced cancer 

Background 

Health spending has continuously increased in Canada over the past few decades. Canada 
is among the top five health spenders in the Organisation for Economic Co-operation 
and Development (OECD) countries, after the United States, the Netherlands, France, 
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and Germany. 1 In the last 3 years, the rate of growth has been 
below inflation and population growth, reflecting government 
efforts to balance budget. Health spending represents 1 1 .2% 
of the Canada's gross domestic product, a share that has fallen 
from a nonsustainable peak of 1 1 .6% in 2009. Weak prospects 
for economic growth combined with fiscal deficits continue 
to create extreme pressure to contain health spending. As 
the percentage of the population aged 80 years and older 
increases, decision-makers will continue to be challenged to 
innovate and reform how health care is provided, including 
ensuring affordability of drugs. 1 

Canada has been a leader in the use of health technology 
assessment (HTA) to assist in the allocation of resources 
within a drug budget. Since its first use in the province 
of Ontario in 1991, HTA has played an increasing role in 
informing public payers' coverage and policy decisions in 
Canada. 2 Over the years, many other jurisdictions around 
the world have adopted HTA frameworks to inform drug 
funding decision-making by considering both clinical and 
cost-effectiveness evidence. 

Conventional HTA criteria pose particular challenges 
for reimbursement decision-making with respect to the 
treatment of advanced cancers. 3 These drugs often have only 
surrogate outcomes, are life-extending rather than curative, 
and are expensive, thus increasing the cost-effectiveness 
ratio. Further, there are equity issues around the weight 
that should be given to the extension of life in patients with 
terminal cancer disease relative to other patients' needs. 
The associated emotional and political context makes these 
decisions more likely to be difficult and controversial. 
To manage this challenge, at least one HTA agency - the 
National Institute for Health and Care Excellence (NICE) 
in the United Kingdom - has developed end-of-life-specific 
HTA criteria. 4 

In Canada, two HTA agencies assess cancer drugs. 
The Institut National d'Excellence en Sante et Services 
Sociaux (INESSS) in the province of Quebec assesses both 
oncology and nononcology drugs, although a specific sub- 
committee focuses on oncology drugs. The pan-Canadian 
Oncology Drug Review (pCODR) assesses cancer drugs 
while the Common Drug Review (CDR) is responsible for 
nononcology drugs in the rest of the country. NICE, which 
operates in England and Wales, has been often used as a 
reference for Canadian decision-makers, at the levels of both 
process and results." 

The objective of this research was to review and compare 
the oncology-specific reimbursement recommendations 
that have been made in Canada alongside those made 



in the United Kingdom by NICE with respect to their 
recommendations records and the influence of clinical and 
cost-effectiveness evidence on the recommendations. 

Methods 

Setting 

Recommendations were reviewed from three agencies: 
the provincial (INESSS) and national (pCODR) oncology 
HTA agencies in Canada, alongside an external reference 
(NICE) that is considered the closest parallel agency 
internationally. 

Canada's national oncology review panel has evolved 
over time. Prior to 2007, the CDR was responsible for 
making funding recommendations regarding oral and 
subcutaneous oncology drugs, while provincial authorities 
made the funding decisions. Intravenous oncology drugs 
were both reviewed and funded by individual or regional 
hospital committees or provincial cancer boards, outside 
of the CDR process. Starting in March 2007, the interim 
Joint Oncology Drug Review (JODR) took responsibility 
for the national review process, inclusive of both oral and 
intravenous oncology drugs. JODR recommendations were 
passed to the provinces for funding decisions. pCODR 
replaced JODR in July 201 1, again making funding recom- 
mendations for both oral and intravenous drugs. Unlike 
JODR, pCODR maintains a website with public posting 
of all reviewed products. Therefore, the national process 
historically consisted of a sequence of three agencies (CDR 
to JODR to pCODR). 

INESSS was established by legislation in 2011, 
succeeding the Conseil du Medicament and the Agence 
d'Evaluation des Technologies et Modes d'lnterventions en 
sante in the province of Quebec. The mission of INESSS is 
to prepare drug funding recommendations for the Minister 
of Health. Previous to INESSS, the Conseil performed 
this function; the legislation that established INESSS 
formalized this process by law within the institution. 
Recommendations from both agencies have been stored in 
the same database. 

In order to manage the scope of the analysis, and to 
facilitate the ability to observe and draw inferences among 
common issues, the analysis focused on disease sites that 
had the most activity with respect to HTA assessments. This 
comprised five disease sites: the most prevalent cancers 
(lung, breast, and colorectal) as well as cancers of the kidney 
and blood/plasma cells (for which many new interventions 
have been introduced in recent years). Furthermore, the 
analysis was limited to drugs for advanced/metastatic stages, 
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or patients with poor prognosis, in order to focus on the 
particular challenge with noncurative, end-of-life drugs. 

The observation period was January 1, 2002 to 
June 1, 2013. All drugs for which final recommendations 
were issued and which met the criteria above, were included 
in the analysis. 

Data abstraction 

The agency websites were used as the primary data source 
for the analysis. Consistent with the HTA agency's individual 
commitments to transparency, only data that were formally 
disclosed in the public domain and posted on the agencies' 
websites were included in the analysis. (Note: there was 
one exception - the review of bevacizumab for metastatic 
colorectal cancer in Quebec was never published on the 
INESSS website but was included in the analysis because 
the information was in the public domain). Specifically, the 
documents retrieved for the analysis were: 

• INESSS: Avis au Ministre 8 

• Rest of Canada: the Canadian Expert Drug Advi- 
sory Committee Final Recommendation from the 
Common Drug Review (2002-2007), 9 the Executive 
Officer Decision from the interim interprovincial Joint 
Oncology Drug Review (2007-201 1) 1011 and the pCODR 
Expert Review Committee Final Recommendation 
(2011-2013) 12 

• NICE: information from the Appraisal Commit- 
tee only (not information from the Manufacturer's 
Submission). 13 

Some drugs were reviewed for multiple indications. If 
so, then all distinct indications were included in the analysis. 
Some drugs were reviewed multiple times for the same 
indication. If so, then only the most recent recommenda- 
tion was included in the analysis. The following data were 
extracted for each recommendation: 

• efficacy: progression-free survival (PFS) gain 

• efficacy: overall survival (OS) gain 

• incremental cost-effectiveness ratio (ICER) 

• recommendation 

• reason for negative recommendation (if applicable) 

• arrangements between payers and manufacturers (if 
applicable: risk-sharing, price negotiation, access 
scheme). Note: NICE uses the terminology "pricing 
scheme" to denote any type of manufacturer agreement, 
including product discounts and rebates. 

It is recognized that clinical benefit goes well beyond PFS 
or OS, including quality of life and safety endpoints, among 
others. While these are extremely relevant endpoints in the 



decision-making context of end-of-life cancer treatments, 
they were not included in this analysis. Clinical benefit was 
narrowly defined using efficacy (PFS/OS) under the prior 
assumptions that 1) efficacy was the paramount aspect of 
clinical benefit and 2) the use of PFS instead of OS has been 
controversial among Canadian HTA experts. 

It is worth noting that all three agencies required 
cost-effectiveness evidence as a submission requirement, but 
there was an important difference for INESSS. At INESSS, 
the five factors defined by legislation that must be consid- 
ered when preparing drug funding recommendations for the 
Minister of Health are: therapeutic value, reasonableness of 
price, cost effectiveness, the advisability of entering the drug 
on the list, and its impact on the health and social services 
system. The drug's therapeutic value is a prerequisite to con- 
sideration of the other four criteria. INESSS does not conduct 
the economic assessment when the clinical effectiveness is 
not satisfactorily established. In the rest of Canada and at 
NICE, cost-effectiveness is always assessed regardless of 
clinical benefit. Nevertheless, since clinical uncertainty leads 
directly to economic uncertainty, it is implicit at other agen- 
cies that in the absence of clinical benefit, cost-effectiveness 
is unlikely to be favorable. 

Data analysis 

Every unique recommendation was considered a separate 
observation. Once all recommendations were identified and 
data were abstracted descriptive statistics were performed. 
Factors related to recommendations were counted both within 
each agency and compared across agencies. 

Results 

A total of 90 recommendations were included in the analysis: 
34 from INESSS, 27 from CDR/JODR/pCODR, and 29 from 
NICE. The distribution by disease site and by HTA agency is 
portrayed in Table 1. INESSS conducted the highest number of 
reviews; INESSS has provided funding recommendations con- 
tinuously throughout the time period and it was charged with 
reviewing all drugs subject to funding by the Regime General 
d' Assurance Medicament. For the rest of Canada, there were 
gaps for two reasons. There was an absence of intravenous drug 
reviews before 2007 (during the CDR era, when only oral drugs 
were reviewed). Further, during the JODR era, reviews were 
conducted but not necessarily posted (as there was no formal 
JODR website). At NICE, there was no formal requirement 
to review all newly licensed drugs/indications. In four cases, 
reviews had been initiated but were terminated before comple- 
tion, because the manufacturer did not submit evidence. 
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Table I Eligible drugs and indications 



Drug by disease site 



Indication 



INESSS CDR/JODR/pCODR NICE 

(n=34) (n=27) (n=29) 



Lung (mNSCLC) 



Bevacizumab 


mNSCLC (first line) 




NR 


NR 


Erlotinib 


mNSCLC (second-third line) 




V 


V 


Erlotinib 


mNSCLC (maintenance) 




NP 




Pemetrexed 


mNSCLC (second line) 


V 


V 


V 


Pemetrexed 


mNSCLC (first line) 


V 


NP 


V 


Pemetrexed 


mNSCLC (maintenance) 




V 


V 


Gefitinib 


mNSCLC EGFR+ 




NP 


V 


Crizotinib 


mNSCLC ALK+ (first line) 


UR 


V 


UR 


Crizotinib 


mNSCLC (second line) 


NR 


V 


NR 



Colorectal (mCRC) 
Bevacizumab 
Cetuximab 
Cetuximab 
Panitumumab 

Breast (mBC) 

Trastuzumab 

Nab-paclitaxel 

Lapatinib 

Lapatinib 

Eribulin 

Everolimus 
Kidney (mRCC) 

Everolimus 

Pazopanib 

Pazopanib 

Sunitinib 

Sorafenib 

Sorafenib 

Temsirolimus 

Bevacizumab 

Axitinib 
Blood/plasma cell 

Lenalidomide 

Bortezomib 

Bortezomib 

Imatinib 

Dasatinib 

Nilotinib 

Nilotinib 

Rituximab 

Tositumomab 

Ibritumomab tiuxetan 

Bendamustine 



mCRC 

mCRC (first line) 
mCRC EGFR+ KRAS nonmutated 
mCRC EGFR+ KRAS nonmutated 
(second-third line) 

mBC HER2+ (first line) 

mBC (first line) 

mBC HR+ HER2+ (first line) 

mBC HER2+ (second line) 

mBC (third line) 

mBC HR+ combo 

mRCC (second line) 
mRCC (first line) 

mRCC (second line after cytokines) 
mRCC (first line) 

mRCC (second line after cytokines) 
mRCC (second line after sunitinib) 
mRCC (poor prognosis first line) 
mRCC 

mRCC (second line) 

Refractory/relapsed MM 

Refractory/relapsed MM 

MM (initial therapy in patients not 

candidates for ASCT) 

CML (first line) 

CML (second line) 

CML (second line) 

CML (first line) 

NHL (first line) 

NHL (second line) 

NHL (second line) 

NHL (relapse/refractory to rituximab) 



NP 
NR 

V 



V 
V 
V 

V 

UR 

V 
V 
V 

NR 

V 

V 
V 

V 
V 
V 



NR 



NP 

V 

NR 
NP 

V 
V 

V 
V 

NR 

V 
V 

NR 

V 

NR 

V 

NP 
NR 
NR 

V 
V 

NP 
NR 
NR 
NR 
NR 

V 



NR 

V 

NR 

V 

UR 

V 
V 
V 

V 
V 

V 

NR 

V 

NR 
NR 
UR 



Notes: In attempting to conduct this research, there were some important barriers with respect to comprehensiveness of drug assessment. NICE is not required to review 
all drugs, and several reviews were terminated for lack of manufacturer-provided evidence. Prior to the establishment of a common national process exclusive to cancer 
drugs (March 2007), centralized Canadian reviews only occurred for oral agents. During the JODR era, all cancer drugs were reviewed consistently, but recommendations 
were not all posted; the lack of publication of recommendations posed another challenge. pCODR intentionally corrected these issues, with a mandate to be inclusive of all 
new cancer drugs marketed in Canada (irrespective of route of administration) and to establish publicity and transparency as a core value. 

Abbreviations: ALK, anaplastic lymphoma kinase; ASCT, autologous stem cell transplant; CDR, Common Drug Review; CML, chronic myeloid leukemia; EGFR, epidermal 
growth factor receptor; HR, hormone receptor; HER2, human epidermal growth factor receptor 2; INESSS, Institut National d'Excellence en Sante et Services Sociaux; 
JODR, Joint Oncology Drug Review; KRAS, Kirsten rat sarcoma viral oncogene homolog] mBC, metastatic breast cancer; mCRC, metastatic colorectal cancer; MM, multiple 
myeloma; mNSCLC, metastatic non-small-cell lung cancer; mRCC, metastatic renal cell cancer; NHL, non-Hodgkins's lymphoma; NICE, National Institute for Health and 
Care Excellence; NR, not reviewed, NP, reviewed but not posted; pCODR, pan-Canadian Oncology Drug Review; UR, under review. 
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The number of positive versus negative recommendations 
varied widely between agencies. Although the national pro- 
cess reviewed 27 drugs, recommendations were posted for 
only 20 drugs, of which 19 (95%) received positive recom- 
mendations (Figure 1). The missing recommendations were 
all from the JODR era. During the most recent pCODR era, 
six of the seven recommendations were positive (86%). 
INESSS had the next highest number of positive recommen- 
dations, at 77% (Figure 2). NICE had the lowest percentage 
of positive recommendations at 48% (Figure 3). 

Additional details are displayed in Figures 1-3. Canada's 
national process (CDR/JODR/pCODR) yielded the most 
positive recommendations, but these were largely cir- 
cumscribed by a request for a pricing arrangement (45%) 
or restricted criteria (20%). The pricing arrangements 
almost always were intended to address issues with cost- 
effectiveness. For example, during the JODR era, typically 
the recommendation would be negative on the basis of an 
economic value gap, while the funding decision would be 
positive, on the basis of a negotiated agreement between the 
province of Ontario and the manufacturer. pCODR created 
a special category of recommendations that would specify 
that funding is conditional to a negotiated agreement to 
improve "cost-effectiveness" - in fact, this recommendation 
was made for most positive recommendations at pCODR. 
With these options available, there were no negative rec- 
ommendations based on cost-effectiveness. In contrast, 
at NICE, while pricing schemes were sometimes used to 
address cost-effectiveness (31%), a rejection based on cost- 
effectiveness was more common (48%). At INESSS, the 
majority of recommendations had no restrictions or arrange- 
ments (59%), with the remaining positive recommendations 
under a risk-sharing agreement (18%). 



1 (5%) 



0(0%) 
9 (45%) 

4 (20%) 



" Rejected— lack of clinical 
evidence 

I Rejected— not cost- 
effective 

Recommended— with 
pricing arrangement 

Recommended— restricted 
population 

Recommended— no 
arrangement 



Figure I Types of recommendations - CDR/JODR/pCODR (n=20). 

Note: Seven recommendations from JODR were not posted and could not be 

included in the analysis. 

Abbreviations: CDR, Common Drug Review; JODR, Joint Oncology Drug Review; 
pCODR, pan-Canadian Oncology Drug Review. 




Rejected-lack of clinical 
evidence 



i Rejected-not cost- 
effective 



Recommended-with 
risk-sharing arrangement 
(promising drug) 

Recommended-no 
arrangement 



Figure 2 Types of recommendations - INESSS (n-34). 

Abbreviation: INESSS, Institut National d'Excellence en Sante et Services 
Sociaux. 



Table 2 shows the percentage of positive recommenda- 
tions by disease site. There was no observable consistent 
pattern to the variability observed between disease sites. 

In the metastatic treatment setting, either PFS or OS 
could be used as a primary or secondary outcome in drug 
trials. The relationship between OS gain and a positive 
recommendation was explored (Table 3). Interestingly, the 
majority of positive recommendations across all agencies 
did not include evidence for OS. Moreover, several drugs 
with OS benefit received negative recommendations. The 
positive recommendations were largely based on surrogate 
endpoints, specifically PFS in most cases. HTA agencies 
recognized that crossover limited the opportunity to obtain 
OS. Overall, across all agencies, only a small minority of 
drugs were rejected for lack of clinical evidence: 13% at 
INESSS (four drugs), 5% by CDR/JODR/pCODR (one drug) 
and 3% at NICE (one drug). 

Interagency agreement was assessed by comparing the rec- 
ommendations for drugs/indications combinations that were 
common to all three agencies (Table 4). Of the 14 pairings 
evaluated by the three bodies, only six (43%) had convergent 
(positive) recommendations, while divergence was observed 
in eight cases. Divergence was found for: pemetrexed, 




Rejected-no clinical 
evidence 

] Rejected-not cost- 
effective 

Recommended-patient 
access scheme 

Recommended-no 
arrangements 



Figure 3 Types of recommendations - NICE (n— 29). 

Abbreviation: NICE, National Institute for Health and Care Excellence. 
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Table 2 Percent negative recommendations by disease site 


Agency 


Lung 


Kidney 


Breast 


CRC 


Blood 


pCODR 












% not known 


38% 


0% 


40% 


0% 


60% 


% negative 


13% 


0% 


0% 


0% 


0% 


INESSS 












% negative 


29% 


33% 


0% 


33% 


18% 


NICE 












% negative 


43% 


63% 


100% 


78% 


14% 



Abbreviations: CRC, colorectal cancer; INESSS, Institut National d'Excellence en 
Sante et Services Sociaux; NICE, National Institute for Health and Care Excellence; 
pCODR, pan-Canadian Oncology Drug Review. 



sorafenib, temsirolimus, axitinib, eribulin, cetuximab, 
panitumumab, and dasatinib. Convergence was more com- 
mon when NICE was excluded. Of the 1 8 drug/indication 
combinations reviewed by both INESSS and the national 
process (details not reported), agreement on a positive 
recommendation was made for 16 pairs (89%); the remain- 
ing two drug/indication combinations were rejected by 
INESSS for cost-effectiveness issues while they were 
accepted by CDR/JODR/pCODR with no evidence of pric- 
ing negotiation. 

Discussion 

Funding recommendations and decisions for pharmaceuticals 
have been a focus for attention and even dissent over the 
past few decades, but advanced cancer treatment decisions 
have been the most contentious and troubling, with frequent 
media attention to funding decisions. The three jurisdic- 
tions under examination in this analysis have each evolved 
a process or framework for the evaluation of cancer drugs 
that differs from other products. In most of Canada, there 
is a separate HTA assessment process for the evaluation 
of cancer versus noncancer drugs; in Quebec, the process 
is the same, but there is a separate advisory committee that 
focuses on cancer drugs; and in the United Kingdom, there 
is a separate set of criteria and cost-effectiveness threshold 
for end-of-life (cancer) drugs. 



Table 3 Recommendations with positive overall survival benefit 



Type of 


INESSS 


CDR/JODR/ 


NICE 


recommendation 




pCODR* 




All recommendations 


1 3/34 (38%) 


5/20 (25%) 


10/29 (35%) 


Positive recommendations 


1 2/26 (46%) 


5/19 (26%) 


6/14 (44%) 


Negative recommendations 


1/8 (12%) 


0/1 (0%) 


4/15 (27%) 



Note: *The seven JODR recommendations that were not posted were excluded. 
Abbreviations: CDR, Common Drug Review; INESSS, Institut National 
d'Excellence en Sante et Services Sociaux; JODR, joint Oncology Drug Review; 
NICE, National Institute for Health and Care Excellence; pCODR, pan-Canadian 
Oncology Drug Review. 



When initiating this research, one of the questions was 
the acceptability of surrogate outcomes (specifically PFS) in 
the metastatic/advanced cancer setting. A working hypothesis 
was that despite approval on the basis of surrogate endpoints 
such as PFS, those endpoints would be seldom recognized 
by HTA bodies. When examining the different jurisdictions, 
we found that this hypothesis was rapidly discarded as very 
few drugs were rejected for clinical reasons. This occurred 
even though the majority of cancer drugs did not have evi- 
dence of OS benefit. There are many possible reasons for the 
acceptability of PFS as a surrogate outcome. HTA reviewers 
could be willing to accept the unavoidable clinical uncertainty 
associated with surrogate endpoints given the limitations 
of clinical trial design. In other cases, clinical benefit may 
have been observed for nonefficacy reasons (such as a safety 
advantage of one treatment over the standard of care, which 
could be of much more value than an impact on OS to patients 
with advanced cancer). The current incorporation of patient 
input into HTA reviews suggests that the patient experience 
of safety and quality-of-life issues may predominate in some 
advanced cancers. 

Given that most drugs were considered to provide accept- 
able clinical benefit, cost-effectiveness was left as the most- 
common criterion for a negative recommendation. This was 
not expected as a recent Canadian paper reported that quality 
flaws in economic analyses in oncology drug reimbursement 
submissions rendered them uninformative. 14 Also, previous 
research on CDR and NICE recommendations (not limited 
to oncology drugs) has found that clinical concerns were the 
most compelling aspect of a submission. 7 Once the clinical 
hurdle had been passed the agencies behaved differently. 
In Canada, cost-effectiveness did not influence the recom- 
mendation once clinical factors had been taken into account; 
rather, price remained as one of four statistically significant 
predictors of a recommendation. 15 In the United Kingdom, 
Devlin et al 16 found that cost-effectiveness explained 85% of 
NICE decisions. INESSS recommendations have not been 
studied in previously published research. 

The analysis demonstrated that cost-effectiveness had 
a larger-than-expected role in oncology recommendations. 
This led to a close examination of the optional arrangements 
between payers and manufacturers to address an unfavorable 
cost-effectiveness ratio. 

In Quebec, 12% of drugs were rejected for lack of 
cost-effectiveness and 18% were recommended with some 
risk-sharing arrangements. Risk-sharing agreements were 
introduced in November 2011 to address concerns about 
the rejection of promising cancer drugs. 17 For those "drugs 
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Table 4 Interagency agreement on recommendations 



Drug 


Indication 


INESSS 


CDR/JODR/pCODR 


NICE 


Erlotinib 


mNSCLC (second - third line) 




_|_ 


-1- 1 \A/il" h ^ rr^ n oomonti 
T I VV 1 LI 1 al 1 al 1 gCI 1 ICI 1 L 1 


Pe m et rexed 


mNSCLC (second line) 




-1- l\A/irn a rr^ncTfimfinl"! 

~ l VV 1 LI 1 allallgCllldlLi 


— ( not 1 friQl" firffirtivfil 

1IILVL LUjL dlG^LIYGf 


Pemetrexed 


mNSCLC (maintenance) 






-L 


Everolimus 


mRCC (second line) 


+ 


■I - (with arrangement) 


~\~ (with arrangement) 


Pazopanib 


mRCC (first line) 


+ 


■I - (restricted population) 


■I - (with arrangement) 


Sunitinib 


mRCC (first line) 




-1- f\A/irn a rr^ncTfimfinl"! 

~ l VV 1 LI 1 allallgCllldlLi 


-1- fu/ifn arrancpmpntl 

~ l W 1 LI 1 al lallgdlldllj 


Sorafenib 


mRCC (second line after cytokines) 


- (not cost effective) 


+ 


— (not cost effective) 


Temsirolimus 


mRCC (first line poor prognosis) 


+ 


+ (with arrangement) 


— (not cost effective) 


Axitinib 


mKL-L. (second line) 


+ (with arrangement) 


+ (restricted population) 


— (not cost effective) 


Eribu lin 


mBC (third line) 




■I - (with arrangement) 


— (not cost effective) 


Cetuximab 


mCRC EGFR+ KRAS nonmutated 


+ 


+ (with arrangement) 


— (not cost effective) 


Panitumumab 


mCRC EGFR+ KRAS nonmutated 
(second-third line) 


- (not cost effective) 


+ 


— (not cost effective) 


Imatinib 


CML (first line) 


+ 


+ 


+ 


Dasatinib 


CML (second line) 


+ 


+ 


— (not cost effective) 



Notes: *Based on Cancer Care Ontario guidelines; funded guidelines assumed even if no HTA review. 

Abbreviations: CDR, Common Drug Review; CML, chronic myeloid leukemia; EGFR, epidermal growth factor receptor; HTA, health technology assessment; INESSS, 
Institut National d'Excellence en Sante et Services Sociaux; JODR, Joint Oncology Drug Review; KRAS, Kirsten rot sarcoma viral oncogene homolog; mBC, metastatic breast 
cancer; mCRC, metastatic colorectal cancer; mNSCLC, metastatic non-small-cell lung cancer; mRCC, metastatic renal cell cancer; NICE, National Institute for Health and 
Care Excellence; pCODR, pan-Canadian Oncology Drug Review. 



deemed promising," a risk-sharing agreement recommenda- 
tion could substitute a "do not list" recommendation from 
INESSS when the cost-effectiveness criterion was not met 
or in case of clinical uncertainty. As of October 28, 2013, 
17 cancer drugs were deemed promising by INESSS, with 
recommendation of coverage with evidence development in 
one case and financial arrangements in 16 other cases. 18 Of 
those 17 drugs, six were part of the current study. None of 
the risk-sharing agreements recommended by INESSS were 
accepted by the Minister of Health, who instead decided 
to fund all drugs at their list price under exception status, 
without further patient access delays. Legal and equity issues 
versus the private sector were brought forward to justify the 
status quo by the Minister. There was a clear willingness to 
fund drugs once they met the clinical value criterion, despite 
concerns that may have existed regarding cost-effectiveness. 
The political environment was and remains strongly sup- 
portive of transparent and uniform pricing across all payers - 
mitigating against pricing schemes. Although substantial 
savings could have been achieved through the arrangements 
proposed by INESSS, they were still not introduced by the 
government. Although it has a leading position in providing 
drug accessibility, Quebec is the only province in Canada 
that does not leverage risk-sharing agreements to provide 
drug access to cancer patients at the end of life. 

At Canada's national HTA process, the postrecom- 
mendation jurisdictional funding decision could include 
confidential pricing arrangements. The majority of drugs 



were recommended within the framework of a proposed 
pricing arrangement (45%) or restricted criteria (20%), and 
none were rejected for lack of cost-effectiveness - presumably 
because payers have the opportunity to negotiate pricing 
arrangements to improve the cost-effectiveness ratio. Since 
July 2010, Canadian jurisdictions have been cooperating on 
price negotiations through the creation of the pan-Canadian 
Pricing Alliance (pCPA). 19 A "do not list" recommendation 
from pCODR is considered a "do not negotiate" listing agree- 
ment by the pCPA process. Patient access is highly unlikely 
to occur in this context unless under an exceptional case- 
by-case basis, as adjudicated by the various jurisdictions. Of 
the 21 notices of implementation (as of October 29, 2013) 
issued by pCODR since its inception in July 201 1, no drug 
has been rejected on the basis of lack of cost-effectiveness; 
all drugs that did not meet the cost-effectiveness criteria 
were recommended with funding conditional on improved 
cost-effectiveness. In the absence of a formal product listing 
agreement process, these drugs could be amenable to pCPA 
for an interprovincial negotiation or to any other informal 
negotiation structure that could represent a unique province 
or a group of provinces. It is too early to provide statistics 
on the number of agreements that were successfully negoti- 
ated and that made the drug actually accessible to patients 
in the various provinces. With the inception of pCODR, all 
jurisdictions intend to align with pCODR recommenda- 
tions, although the provinces retain the authority to make 
independent decisions. 
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For most of the CDR's tenure to review oncology 
medicines, price negotiations were not available or considered 
within recommendations, while both the JODR and pCODR 
eras featured the opportunity for product-listing agreements. 
There were important differences that could be responsible 
for changes over time seen within the body of national rec- 
ommendations - differences in mandate, processes, legisla- 
tive framework, expert panel composition, etc. However, it 
was the introduction of price negotiations that truly defined 
the different nature of JODR/pCODR versus CDR recom- 
mendations, and that allowed for such a high percentage 
of positive recommendations conditional on improvement 
of the cost-effectiveness ratio through negotiation with the 
manufacturers. 

In contrast, at NICE, almost half of drugs were rejected 
on the grounds of cost-effectiveness, despite the option to 
negotiate access schemes within the HTA process. Whereas 
no explicit or implicit cost-effectiveness thresholds have 
been introduced in Canada, NICE has a stated cost-effec- 
tiveness threshold range that drugs are expected to meet. If 
a patient access scheme could not be devised to improve the 
cost-effectiveness ratio, then there was an unwillingness to 
make a positive recommendation. In January 2009, NICE 
introduced supplementary advice to accommodate a higher 
cost-effectiveness threshold for oncology drugs that may 
be life-extending for patients with short life expectancy. 
The additional advice applies when the cost-effectiveness 
ratio exceeds the NICE upper end of £30,000 per quality- 
adjusted life-year (QALY) (£35,000 to £45,000), provided 
that the treatment extends life by at least 3 months compared 
with available alternatives. The rationale for this adaptation 
was the assumption that society valued QALYs obtained 
by patients at the end of life more than QALYs obtained by 
other patients. NICE accommodated a higher threshold for 
end-of-life drugs, but this higher threshold must still be met 
for a positive recommendation. 4 

More recent rejections by NICE of various oncology 
products highlighted the problems facing the launch of new 
cancer medicines in the United Kingdom. These negative 
decisions were noted in many other countries around the 
world, promoting the perception that UK patients were being 
denied access to oncology medicines. What is less well- 
known internationally is that the UK National Health Service 
(NHS) offered an alternative route to fund cancer medicines: 
the national Cancer Drugs Fund (CDF). In contrast to their 
well-publicized rejections by NICE, many drugs were fully 
funded for use via the CDF, introduced in England in April 
2011. This fund was established in order to enable patients 



to access the cancer drugs their doctors think will help them, 
even if funding is not recommended by NICE. Its establish- 
ment was confirmed by the UK government's coalition 
agreement in 20 10. 21 The objectives of the CDF, as set out 
by the UK government are: 

• to provide maximum support to NHS patients; 

• to put clinicians and cancer specialists at the heart of 
decision-making, consistent with the government's wider 
policy of empowering health professionals and enabling 
them to use their professional judgment about what is 
right for patients; and 

• to act as an effective bridge to the government's aim of 
introducing a value-based pricing system for branded 
drugs in 20 14. 22 

The funding system was initially set up to run for a 
period of 3 years, from April 201 1 to March 2014, but the 
government has announced an extension for another 2 years 
to make the fund available until March 2016. The CDF is 
allocated £200 million per year. 

The scope of the funding includes: 1) drug/indication 
combinations appraised by NICE and not recommended on 
the basis of cost effectiveness, or where the recommenda- 
tions restrict access to a smaller group of patients than the 
specifications set out in the marketing authorization; and 
2) drug/indication combinations for which NICE has not, 
or not yet, issued guidance. 22 The establishment of the CDF 
was a political response to increasing public and healthcare 
professional pressures about access to new cancer treatments. 
Setting aside £200 million per year out of the healthcare bud- 
get highlighted the government's additional willingness to pay 
for cancer therapies above and beyond the NICE threshold 
and made predictable budgeting an acceptable alternative to 
the cost/QALY approach. This may not be a viable solution 
in the long term for several reasons: the budget may end up 
being insufficient as other treatments are approved; it cre- 
ates inequities in access by being available only to patients 
in England (not Wales); and it seems to value the QALYs 
obtained for cancer patients more than those obtained for 
other patient populations who have started advocating for 
special funds (eg, patients with rare disease). Overall, a NICE 
positive recommendation still remains the optimal way to 
ensure all eligible patients in England and Wales have routine 
access to therapy. 

In summary, the lack of rejection based on clinical 
grounds suggested that the evidence packages submitted 
to HTA agencies were sufficient to address questions of 
therapeutic value in advanced cancers, across all jurisdictions. 
In contrast, disparate findings were observed for the role of 
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cost-effectiveness. Lambda, the cost-effectiveness threshold, 
was found by Canadians involved in oncology reimbursement 
to be only one of many factors that should inform funding 
decisions - alongside consideration of broader factors, includ- 
ing formal guidance on weighting both evidence and values. 3 
Despite this broad vision expressed by a variety of oncology 
experts, Canadian HTA agencies are narrowly focusing on 
price as a primary mechanism to improve cost-effectiveness 
and yield more positive funding recommendations. 

The UK is also exploring alternative ways to widen the 
range of factors considered in decision-making to improve 
NHS patient access to effective and innovative drugs 
while ensuring they are available at a price that reflects the 
value they bring. The Department of Health proposed to 
introduce a value-based pricing (VBP) approach that would 
have succeeded the Pharmaceutical Price Regulation Scheme 
(PPRS) expiring at the end of 20 13. 23 Abolishment of the CDF 
was one of the objectives of this pricing reform. However, 
the UK government moved away from the proposed VBP 
approach by reaching an agreement on a new PPRS with 
the Association of the British Pharmaceutical Industry. By 
making the "P" in VBP no longer applicable, the 2014 PPRS 
led to the recommendation by NICE of a value-based assess- 
ment (VBA) approach that adds two factors to their appraisal 
methods: the burden of illness and the wider societal benefit. 24 
How these proposed changes will eventually impact decisions 
in the United Kingdom remains to be seen. 

Conclusion 

This Canada-UK comparison revealed the lack of interagency 
consistency in recommendations, relative importance of cost- 
effectiveness to oncology recommendations, and the diversity 
of approaches to address unfavorable cost-effectiveness: an 
exclusion of price negotiation from both the drug review 
and funding process (INESSS), a reliance on confidential, 
postreview price negotiation to achieve cost-effectiveness 
(pCODR), a fixed cost-effectiveness threshold that requires 
either a pricing scheme during the review process or a high 
level of rejection based on failure to meet the threshold 
(pre-2014 NICE), or the explicit incorporation of nonmon- 
etary factors to broaden the cost-effectiveness threshold 
(value-based assessment proposed by NICE in 2014). 

One reason for the findings is that each jurisdiction has 
different HTA objectives and processes. The low rate of con- 
vergence for positive outcomes on the same drug/indication 
(43%) supported the fact that the three agencies had very dif- 
ferent strategies to deal with clinical uncertainty or unfavor- 
able cost-effectiveness. There was an apparent relationship 



between the pressure of negotiating solutions to improve 
ICERs and the outcome of the HTA recommendation. The 
Canadian processes put most of the pressure on the payers 
(in Quebec) and manufacturers (in the rest of Canada) to 
reach a financial arrangement. On the other end, the NICE 
approach was more in favor of risk-sharing agreement and 
price negotiation during the HTA review to meet an agency- 
defined ICER threshold, and thus NICE itself bore most of 
the pressure. 

There were many more differences betweenjurisdictions 
than simply their perspectives on cost-effectiveness within 
the HTA assessment process. There were important and sub- 
stantial differences with respect to mandate, process, legisla- 
tive setting, etc. Even the evidence base presented to HTA 
agencies may differ betweenjurisdictions. 25 These obviously 
all contributed to the observed findings. The future for the 
cost-effectiveness criterion within an HTA assessment, and 
within a funding decision-making framework, will depend 
on the jurisdiction under consideration, its current mandates 
and processes, and the tools available to achieve its goals. 
There are multiple levers, outside of HTA evaluation, that 
can be used to address high ICERs: coverage with evidence 
programs, value -based pricing, adaptive licensing, etc, in 
addition to the price negotiations that were observed in this 
research. In all jurisdictions, the review process and funding 
decisions for pharmaceuticals should have governance struc- 
tures that are fair, objective, transparent, and accountable to 
patients, payers, and the public. 

In this analysis, the primary reason for the observed 
divergence between agency recommendations appeared to 
be the availability of mechanisms in each jurisdiction to 
address cost-effectiveness subsequent to the HTA assessment 
process. Furthermore, caution is needed when interpret- 
ing cross-agency comparisons between HTA agencies, as 
recommendations may not correspond directly to subsequent 
funding decisions and actual patient access. This may be 
a concern, given the high profile internationally of HTA 
assessments conducted by the reviewed agencies. 
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